Claude/test spec model structure cgfvxp#76
Merged
Conversation
A data-provider over build/test/test.json — not a test runner. Reads the shared corpus and yields normalized Entry records (tagged input: in|args|ctx with ctx>args>in precedence; tagged expect: value|error|match|absent), plus pure comparison helpers (matchval, equal, equalStrict, structMatch, errorMatches). Execution and assertion stay with the agent-written tests that consume it. - PROVIDER.md: language-neutral data model + API + helper semantics. - AGENTS.md: guidance for agents writing tests — per-function input->call mapping table and the null:false function list (equalStrict). - ts/ is canonical; python/go/rust are faithful ports. All four smoke-check identically: 1325 entries (value 1181, absent 84, error 59, match 1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
…pp, java, rust) Adds the test-provider library for 7 more languages and makes Rust dependency-free. Each is stdlib-only or hand-rolls a minimal JSON parser (java/c/cpp/rust own parser; rust drops serde_json). All independently smoke-verified here against the canonical numbers: 1325 entries (value 1181, absent 84, error 59, match 1), getpath/basic[0] = 42. Build artifacts (rust/target, *.class) gitignored. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Stdlib-only ports (swift hand-rolls an order-preserving JSON parser since Foundation reorders keys; dart uses dart:convert; csharp uses System.Text.Json). NOT locally executed — these toolchains are absent in this environment. Each was self-reviewed against ts/provider.ts and its normalization validated against the corpus via a Python replica (reproduces 1325 / value 1181 / absent 84 / error 59 / match 1). To be run under each port's own test job in CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Stdlib-only (std.json; ObjectMap preserves key order). NOT locally executed — zig toolchain absent here; faithful port self-reviewed against ts/provider.ts and corpus-validated via a Python replica (1325 / value 1181 / absent 84 / error 59 / match 1). API targets Zig 0.13 per the zig port. "/re/" uses a minimal matcher (no std regex) — does not affect the key-presence-based counts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Both hand-roll a JSON parser (kotlin: LinkedHashMap, base-only; haskell: order-preserving assoc-list reader adapted from its existing runner, GHC base only — no kotlinx/gson/aeson). NOT locally executed — toolchains absent here; self-reviewed against ts/provider.ts and corpus-validated via a Python replica (1325 / value 1181 / absent 84 / error 59 / match 1). Haskell "/re/" is a documented substring approximation (base has no regex). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Pure-Lua recursive-descent JSON parser (no dkjson/lfs); ordered-object representation + JSON_NULL sentinel so key order and null-vs-absent are preserved. NOT locally executed — no lua toolchain here; corpus-validated via a Python replica (1325 / value 1181 / absent 84 / error 59 / match 1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
OCaml reuses an order-preserving JSON reader adapted from its existing runner (stdlib only, no yojson); Elixir hand-rolls a pure parser with an ordered Obj struct + :null atom (no Jason/:json). NOT locally executed — toolchains absent; corpus-validated via Python replicas (1325 / value 1181 / absent 84 / error 59 / match 1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Scala 3, stdlib-only: own order-preserving JSON ADT + minimal parser (no play-json/circe). NOT locally executed — scala toolchain absent; corpus-validated via a Python replica (1325 / value 1181 / absent 84 / error 59 / match 1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Clojure: hand-written JSON parser (no data.json/cheshire); objects carry insertion order in :order metadata; ::null sentinel keeps null distinct from absent. NOT locally executed — clojure toolchain absent; corpus-validated via a Python replica (1325 / value 1181 / absent 84 / error 59 / match 1). Completes all 22 ports. PROVIDER.md now records the full language matrix, the dependency-free / order-preserving JSON guarantee, and which ports are locally verified vs corpus-validated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Installed the missing toolchains (lua, ocaml, ghc, elixir, dotnet, zig,
dart, kotlin, scala, clojure, swift) and ran every port's smoke: ALL 22
reproduce the canonical numbers (1325 / value 1181 / absent 84 / error 59
/ match 1). No longer any "not executed" ports.
Fixes found by actually running them:
- csharp: invalid XML comment in StructProto.csproj ('--' inside a
comment) broke the build — reworded.
- swift: top-level smoke code must live in main.swift (Swift only allows
top-level statements there) — renamed smoke.swift -> main.swift.
Docs:
- RUNNING.md: exact verified run command per language + corpus-path
convention + toolchain versions + two known packaging nits (swift
main.swift, clojure ns/path-depth mismatch with a documented run recipe).
- PROVIDER.md: verification note now records all 22 run-verified; still
intentionally out of CI (prototype).
- .gitignore: cover bin/obj, cmi/cmo, jars so smoke runs stay clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
Evaluates the test/proto prototypes against the bar of a general test-spec-provision library. Verdict: proven core, not yet a drop-in library. Documents what's already library-grade (22-way parity, dep-free, centralized helpers) and the blocking gaps — invoke-mapping living in AGENTS.md prose, per-group null-mode absent from the corpus, args/ctx and primary.check unreachable, no clone-on-read, helpers unproven, hardcoded corpus discovery, no provider-level parity check, packaging. Proposes concrete fixes, centered on pushing invoke-mapping + null-mode into the aontu test-spec model (converges with TESTSPEC_MODEL.md), and a phased order. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn6v25EtY7Kss941hkMhrn
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.